Bottleneck Conditional Density Estimation
نویسندگان
چکیده
We propose a neural network framework for high-dimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input x and target y, where both are highdimensional. The key to effectively train BCDEs is the hybrid blending of the conditional generative model with a joint generative model that leverages unlabeled data to regularize the conditional model. We show that the BCDE significantly outperforms the CVAE in MNIST quadrant prediction benchmarks in the fully supervised case and establishes new benchmarks in the semi-supervised case.
منابع مشابه
Bottleneck Conditional Density Estimators
We propose a neural network framework for highdimensional conditional density estimation. The Bottleneck Conditional Density Estimator (BCDE) is a variant of the conditional variational autoencoder (CVAE) that employs layer(s) of stochastic variables as the bottleneck between the input x and target y, where both are high-dimensional. The key to effectively train BCDEs is the hybrid blending of ...
متن کاملFast Nonparametric Conditional Density Estimation
Conditional density estimation. The idea of conditional density estimation is to construct a density estimate f̂(y|x) for a dependent variable y, conditional on a vector of variables x. This can be seen as a generalization of regression, where instead of estimating the expected value E(y|x) alone, we instead model the full density. This is especially important for multi-modal densities, where th...
متن کاملConditional Density Estimation via Least-Squares Density Ratio Estimation
Estimating the conditional mean of an inputoutput relation is the goal of regression. However, regression analysis is not sufficiently informative if the conditional distribution has multi-modality, is highly asymmetric, or contains heteroscedastic noise. In such scenarios, estimating the conditional distribution itself would be more useful. In this paper, we propose a novel method of condition...
متن کاملSemi-Supervised Least-Squares Conditional Density Estimation
---Conditional density estimation is an useful alternative to regression to learn an input-output relationship under multi-modality, asymmetry, and heteroscedasticity. The supervised learning method called least-squares conditional density estimation (LSCDE) is the state-of-the-art method that directly estimates the conditional density using a linear model. In this paper, we extend the supervis...
متن کاملSimilarity Based Smoothing In Language Modeling
In this paper, we improve our previously proposed Similarity Based Smoothing (SBS) algorithm. The idea of the SBS is to map words or part of sentences to an Euclidean space, and approximate the language model in that space. The bottleneck of the original algorithm was to train a regularized logistic regression model, which was incapable to deal with real world data. We replace the logistic regr...
متن کامل